Optimization tool for auditory devices

ABSTRACT

An optimization system for testing a patient&#39;s hearing comprises a controller, an ear piece, and a memory. The controller: provides a series of tones to the ear piece; receives feedback from the patient between each tone; generates a data point to be used in an audiogram after receiving each feedback; after each data point is generated, computes a statistical distribution based on the generated data points; identifies an area of the statistical distribution most in need of additional data; and selects a subsequent tone to provide in the series of tones. Each feedback indicates whether the respective tone was detected or not detected, and each data point is based on the respective feedback. Each subsequent tone provided in the series of tones is a tone represented in the area of the statistical distribution most in need of additional data at the time of selection.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application comprises a continuation of U.S. application Ser. No.16/848,517 filed Apr. 14, 2020, which is a continuation of U.S.application Ser. No. 16/294,912 filed Mar. 6, 2019 (U.S. Pat. No.10,617,334), which incorporates by reference and claims the benefit ofpriority to U.S. Provisional Application No. 62/639,489, filed on Mar.6, 2018, the disclosures of which are incorporated herein by theirentireties.

BACKGROUND OF THE INVENTION

The present invention relates systems and methods for optimizingparameters of hardware for audiological devices. More specifically, thepresent invention relates to systems and methods in which acoustic wavesare transformed into electrical signals in a device, and the settings ofthe device are tailored to the individual.

Programming hardware for audio signals is complicated due to thecomplexity of audio signals. In addition to the basic problemsassociated with reproducing a constantly changing sound comprised of anoverlapping collection of various pitches and amplitudes, problems arecompounded by signal to noise issues, threshold hearing variances acrossa wide range of the spectrum in which humans can hear, and other uniquefactors. With such a complex variable set, or from another perspective,such a wide optimization space, it is difficult for a user or operatorto arrive at an optimized setting.

For example, cochlear implants include technology that transformscomplex auditory waves into pulses to be sent to a plurality of channelson the inner cochlea of a patient in order to stimulate the neurons onthe select channels. The process of transforming auditory waves intoelectronic signals requires the transformation of a multitude ofinformation including frequency, amplitude, and voltage among backgroundnoise and environments into an electrical signal to recreate hearing.

Cochlear implants are neural prostheses that help severely-to-profoundlydeaf people to restore some hearing. Physically, three components can beidentified: the speech processor with its transmission coil, thereceiver and stimulator, and the cochlear implant electrode array. Thespeech processor receives sound from one or more microphones andconverts the sound into a corresponding electrical signal. While thehearing range of a young healthy human is typically between 0.02 and 20kHz, it has been assumed for coding of acoustic information in cochlearimplants that most of the information used for communication is in thefrequency range between 0.1 and 8 kHz. The frequency band from 0.1 to 8kHz is divided into many smaller frequency bands of about 0.5 octaveswidth. The number of small frequency bands is determined by the numberof electrodes along the electrode array, which is inserted into thecochlea. Each frequency band is then treated by a mathematicalalgorithm, such as a Hilbert transform that extracts the envelope of thefiltered waveform. The envelope is then transmitted via an ultrahighfrequency (UHF) connection across the skin to a receiver coil, which wassurgically implanted behind the ear. The envelope is used to modulate atrain of pulses with a fixed pulse repetition rate. For each of theelectrodes, a train of pulses with fixed frequency and fixed phase isused to stimulate the cochlear nerve. Multiple algorithms have beenimplemented to select a group of 4-8 electrode contacts for simultaneousstimulation.

Damage of cochlear neural structures can result in severe deafness.Depending on the neural degeneration in the cochlea performance, theperformance of a cochlear implant user may vary. Changes that occurinclude the demyelination and degeneration of dendrites and neuronaldeath. The neuronal loss can be non-uniform and results in “holes” ofneurons along the cochlea. Holes lead to distortion of the frequencymaps, which affects speech recognition. Caused by changes in myelinationand synapse size, changes in firing properties of the nerve weredescribed such as prolonged delay times and changed refractory periods.In the brainstem and midbrain the neuronal connections appear to remainintact. However, a decrease in neuron size, afferent input, synapse sizeand density can be detected. Neural recordings reveal a change inresponse properties that adversely affect temporal resolution such aselevated thresholds, de-synchronization, increased levels of neuraladaptation, increased response latencies. A loss of inhibitoryinfluences has been described. At the cortex, spatially larger corticalactivation was seen with (PET). The findings support a plasticreorganization and more intense use of present auditory networks.

A conventional cochlear implant includes a speech processor thattransforms the acoustic waves received on the microphone into anelectrical signal that stimulate the implanted electrode array, andconsequently, the auditory nerves. Auditory waves are a complexsummation of many different wave forms, and the processor decomposes thecomplex auditory signal received on the microphone into discretecomponent frequencies or electrical pulses to be sent to the auditoryneurons through the electrodes. Nerve fibers in the vicinity of theelectrodes are stimulated and relay information to the brain. Loudsounds produce high-amplitude electrical pulses that stimulate a greaternumber of nerve fibers, while quiet sounds produce low-amplitude pulseseffected a lesser number of nerve fibers. Different variables within thesoftware on the processor affect the output of the cochlear implantspeech processor.

To activate the cochlear implant, an audiologist tunes the levels andstimulation parameters of the speech processor so that the sounds pickedup by the microphone are heard at the individual's ideal loudness level.Initially, the audiologist stimulates the implant's channels orelectrode pairs with small electrical pulses to test whether the userhears the stimulus. Over the course of subsequent sessions, theaudiologist performs a series of tests to understand the user'slistening needs. The user's cochlea is tuned to perceive differentpitches depending on the area being stimulated. During the sessions, theaudiologist stimulate the implant's channels to simulate pitchdifferences. The audiologist will also vary the electrical current oneach channel to find the most comfortable loudness level. Theaudiologist may also take threshold measurements to understand theuser's softest level audible on each channel. The audiologist ultimatelygenerates a map that is downloaded to the speech processor to enable theprocessor to appropriately adjust volume levels based on theindividual's needs.

With cochlear implants and other hearing devices, each patient isunique. Following implantation, changes occur that can affectperformance of the device. Changes include genetic disorders, iatrogenicprocedures, ototoxic drugs, or loud noise exposure. The user's hearingwill change over time, requiring additional visits to the audiologist inorder to rerun the tests and adjust the map accordingly.

Additionally, hearing devices other than cochlear implants, such asrecent hearing aid technology, may require programming based onaudiological feedback during testing to achieve optimal results. Thelatest generations of hearing aids and other “hearables” includeparameter settings for amplification, compression, noiserejection/cancellation, etc. Being able to fine tune each of theparameters, in each ear, provides even greater flexibility in theoptimization of these devices. However, the complexity created by themany parameters can be a challenge for manual tuning.

Accordingly, there is a need for an optimization system for effectivelyadjusting a large number of parameters of a hearing device whileaccounting for a variety of hearing situations.

SUMMARY OF THE INVENTION

The optimization system of the present application allows a patient tointuitively define parameter settings for a large number of parametersin association with a variety of environments. The optimization systemincludes a first module, a second module, and a third module. It isunderstood that the first through third modules may be integrated into asingle program or be provided in a fewer or greater number of modules.The first module collects and analyzes a wide range of patient feedbackas input data to determine ranges for each parameter tailored to thecochlea of the patient. The second module includes a plurality of userinterfaces that prompt the patient to select a preferred set of datapoint(s) corresponding to a specific pitch and/or frequency. The datapoints initially provided to the patient in the second module are withinthe specific ranges provided as output from the first module. Within thesecond module, one or more user interfaces allow the patient to comparea large number of parameter settings against one another in a singletrial.

The third module includes a database on which a reference bank of soundsor environments that identify frequency content associated with eachacoustic environment is stored. Using wavelet scattering transforms, aclassifier algorithm determines the frequency content of each acousticenvironment. Alternative methods such as traditional Fourier transformsor spectrograms may be used. Support vector machine (SVM), K-clusteringmechanisms, or any type of clustering methods is used to group theinformation contained in each acoustic environment to create a referencebank of sounds (i.e., noisy restaurant, beach, and metro station).

In one embodiment, a system for controlling parameter settings of anauditory device includes: an auditory device processor; an auditorydevice output mechanism controlled by the auditory device processor, theauditory device output mechanism including one or more modifiableparameter settings; an auditory input sensor that detects anenvironmental sound and communicates with the auditory device processor;a database in communication with the auditory device processor, thedatabase pairing each of a plurality of sets of parameter settings witha corresponding sound profile; a memory in communication with theprocessor and including instructions that, when executed by theprocessor, cause the processor to: receive the environmental sounddetected by the auditory input sensor; analyze a frequency content ofthe environmental sound; compare the frequency content of theenvironmental sound with the sound profiles stored in the database and,in response to the comparison, select one of the sound profiles; andautomatically adjust the parameter settings of the auditory deviceoutput mechanism to match the set of parameter settings associated withthe selected sound profile.

In some embodiments, when the auditory device processor analyzes afrequency content of the environmental sound it uses a waveletscattering transform to analyze the frequency content of theenvironmental sound. In other embodiments, when the auditory deviceprocessor analyzes a frequency content of the environmental sound ituses a Fourier transform to compute the frequency content of theenvironmental sound.

In some embodiments of the systems herein, the sound profiles areclustered by similarities. The frequency content of the environmentalsound may include one or more properties selected from the groupcomprising a signal-to-noise ratio, an amplitude range, and a pitchrange. The one or more properties selected from the group comprising asignal-to-noise ratio, an amplitude range, and a pitch range of thefrequency content of the environmental sound matches the correspondingone or more properties selected from the group comprising asignal-to-noise ratio, an amplitude range, and a pitch range of one ofthe sound profiles.

In some examples of the system, the auditory device output mechanism isan electrode of a cochlear implant and the auditory input sensor ismicrophone of a cochlear implant.

In some examples of the system, the auditory device output mechanism isspeaker of a hearing aid and the auditory input sensor is microphone ofa hearing aid.

Each set of the plurality of sets of parameter settings may includeamplification settings, compression settings, and directional noiserejection settings.

In some examples of the system, each sound profile is associated with astored geolocation, the system further comprises a location sensingmechanism in communication with the auditory device processor, and whenthe processor compares the frequency content of the environmental soundwith the sound profiles stored in the database and, in response to thecomparison, selects one of the sound profiles, the processor furthercompares a present geolocation of the auditory device output mechanismidentified by the location sensing mechanism with the storedgeolocations.

An objective of the present design is to provide a user-friendlyoptimization system for adjusting a variety of parameters of a hearingdevice. In some examples, the hearing device is a cochlear implant. Inother examples, the hearing device is a hearing aid. In other examples,the hearing device is another hearable device.

An objective is to provide a system for automatically controllingparameter settings of an auditory device such that the deviceautomatically updates its settings in response to recognizing theauditory environment in which it is being used.

An objective is to improve the performance of auditory devices across awide range of audio environments by enabling real-time adaptation of thesetting of the device.

Additional objects, advantages and novel features of the examples willbe set forth in part in the description which follows, and in part willbecome apparent to those skilled in the art upon examination of thefollowing description and the accompanying drawings or may be learned byproduction or operation of the examples. The objects and advantages ofthe concepts may be realized and attained by means of the methodologies,instrumentalities and combinations particularly pointed out in theappended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, features, and advantages of the presentdisclosure set forth herein will be apparent from the followingdescription of particular embodiments of those inventive concepts, asillustrated in the accompanying drawings. Also, in the drawings the likereference characters refer to the same parts throughout the differentviews. The drawings depict only typical embodiments of the presentdisclosure and, therefore, are not to be considered limiting in scope.

FIG. 1 is a schematic illustrating the components of the optimizationsystem of the present disclosure in use with a cochlear implant.

FIG. 2 is a block diagram illustrating the interaction between thefirst, second, and third modules of the optimization system of FIG. 1 .

FIGS. 3A-3E are charts illustrating the potential variance provided bymodifying the parameters of an example coding strategy of a cochlearimplant that may be used with the optimization system of FIG. 1 .

FIG. 4 is a representation of a user interface of an audiometrygathering screen.

FIG. 5 is a representation of a first user interface of a sound testingscreen.

FIG. 6 is a representation of a second user interface of a sound testingscreen.

FIG. 7 is a representation of a rating spectrum used in connection withthe second user interface of FIG. 6 .

FIG. 8 is a representation of a third user interface of a sound testingscreen.

FIG. 9 is a graphic representation of a cluster of sounds of a referencebank.

FIG. 10 is a block diagram illustrating the an alternative optimizationsystem.

FIG. 11 is a representation of a user interface of the optimizationsystem of FIG. 10 .

FIGS. 12-15 are representations of user interfaces illustrating the datacollection of the optimization system of FIG. 10 .

DETAILED DESCRIPTION

The present application provides an optimization system that optimizesthe parameters of an auditory device based on the individual's specificneeds to improve the user's ability to hear.

FIG. 1 is a block diagram illustrating an example system 100 forperforming various activities involving the activation, modulation,and/or blockage of neurons within the brain using an audio device suchas a cochlear implant or hearing aid. As illustrated, the system 100includes a neurostimulation device (NSD) 102. The NSD 102 may beimplantable (e.g., below the skin), or alternatively, may be some typeof external device, such as a cochlear implant device or a hearing aiddevice. Primary example used herein is a cochlear implant, although thesystem 100 may apply to a hearing aid or other hearable device.

In the example shown in FIG. 1 , the NSD 102 transforms acoustic wavesinto electrical impulses. An auditory input sensor 104 such as amicrophone on the NSD 102 captures the acoustic wave, and a controller106 including an auditory device processor 108 deconstructs the acousticwave and utilizes a pulse generator 110 to generate discrete electricalpulses that are then provided to an auditory device output mechanismsuch as an electrode array on the cochlea. Specifically, the pulsegenerator 110 generates electrical impulses (“pulses”) in specificpatterns for electrical stimulation of nervous tissue of the cochlea.Stated differently, the pulses generated by the pulse generator 110 areapplied in specific patterns to specific regions and/or portions of thenervous system to deliver neurostimulation. The pulse generator 110 maybe electrically coupled to electrodes 112 and 114 via one or more leads116 and 118, respectively, and thereby provide neurostimulation to thespecific regions of the nervous system. The pulses generated by thepulse generator 110 are conducted through the one or more leads 116 and118 and terminated in the electrodes 112 and 114 generally implanted inthe tissue of the nervous system. In another embodiment where theauditory device is a hearing aid, the auditory device output mechanismmay be a speaker.

The auditory device processor 108 on the controller 106 controls thepulse generator 110 to deliver electrical pulses (i.e.,neurostimulation) according to a selected stimulation parameter set(e.g., pulse amplitude, pulse width, pulse frequency, etc.) and/or otherinstructions to applicable regions of the nervous system.Neurostimulation programs or coding strategies based on variableparameters that are used in the delivery of neurostimulation therapy(i.e., stimulation) may be stored in a memory 120, in the form ofexecutable instructions and/or software, for execution by the auditorydevice processor 108. The auditory device or NSD 102 may also include aglobal positioning system (GPS) chip 121 and a database of referencesound profiles 123, which may be utilized in the programming stored onthe memory 120 as described below.

In some embodiments, the controller 106 may contain a machine-learninglogic unit (“MLU”) 122 that is trained to perform machine-learningoperations involving the generation of various predictions that may beused to optimize the functionality of the NDS 102 and/or initiate andoptimize neurostimulation therapy provided to a patient via the NDS 102.The MLU 122 may process data received from users interacting with theNDS 102 when generating such predictions. Although the controller 106 isillustrated as being included within the NSD 102, in some embodiments,it may be implemented in a computing device that is separate from theNDS 102. In such an embodiment, the controller 106 may communicate withthe NSD 102 remotely, such as through a communications network 124,which may be a telecommunications network, the Internet, an intranet, alocal area network, a wireless local network, radio frequencycommunications protocol, or any other type of communications network, aswell as combinations of networks.

The NSD 102 may be communicatively connected to an optimization device126 and/or an audiologist device 128 locally or via the communicationsnetwork 124 to receive input that may be processed to optimizeneurostimulation therapies and/or optimal functions of the NDS 102. Eachof the optimization device 126 and the audiologist device 128 providesuser interface(s) that enable a patient or user to provide the input(e.g., data) to the NSD 102 that defines, qualifies, and/or quantifiesaspects of the neurostimulation therapy provided by the NDS 102. Morespecifically, variables of the equations that are part of computerprogram stored in the memory 120 of the NSD 102 are set by theoptimization interface and/or the audiologist interface of theoptimization device 126 and the audiologist device 128, respectively.Each of the devices 126, 128 may include a processor-based platform thatoperates on an operating system, such as Microsoft® Windows®, Linux®,iOS®, Android®, and/or the like that is capable of executing and/orotherwise generating the interfaces.

The user or operator of the optimization device 126 works with thepatient wearing the NSD 102 to gather user feedback in response to audiotests as shown in FIGS. 4-8 and described below. The user interfaces ofthe optimization device 126 can be coupled to memory 130 that can storeprogram instructions 132 to run the optimization system. Further, thememory 130 may also store communication instructions 134 to facilitatecommunicating with one or more additional devices, one or morecomputers, and/or one or more servers. The memory 138 may includegraphical user interface instructions 136 to facilitate graphic userinterface processing.

The audiologist operates the device 128 to directly adjust theprogramming or instructions on the memory 120 of the NSD 102.Specifically, the audiologist may provide input in the form of a set ofstimulation parameters that define various parameters, such as pulseamplitude, pulse width, pulse frequency, etc., any of which may be usedto automatically determine a specific neurostimulation therapy (e.g.,parameter space) for a particular patient. Based on such input, thecontroller 106 logically directs the pulse generator 110 to modifyinternal parameters and vary the characteristics of stimulation pulsestransmitted to the nervous system. The audiologist may interact with theoptimization device 126 to provide feedback regarding the success of thesimulation (e.g., better, same, or worse) in comparison to previousneurostimulation therapies, to modify parameters of the currentsimulation, etc.

Each of the above identified instructions and applications cancorrespond to a set of instructions for performing one or more functionsdescribed herein. These instructions need not be implemented as separatesoftware programs, procedures, or modules. The memory 130 can includeadditional instructions or fewer instructions. Furthermore, variousfunctions of the system 100 may be implemented in hardware and/or insoftware, including in one or more signal processing and/or applicationspecific integrated circuits.

In one example, the memory 120 includes stored instructions that, whenexecuted by the auditory device processor 108, cause it to deconstructacoustic waves into discrete electrical signals and to generateelectrical pulses through the pulse generator. In one example, U.S. Pat.No. 9,717,901 discloses a frequency-modulated phase coding (FMPC)strategy to encode acoustical information in a cochlear implant 102. Theentirety of the disclosure provided by U.S. Pat. No. 9,717,901 isincorporated herein. The FMPC strategy utilizes the following equationthat describes the relationship between the sound level at the outer earcanal and the corresponding rate of action potentials that can berecorded from a single auditory nerve fiber. This function is expressedbelow and includes cochlear nonlinearities and depends on five criticalparameters: a spontaneous rate (a₀), a maximum rate (a₁), a thresholdfor stimulation (a₂), a level for nonlinear behavior (a₃), and a valuedescribing the slope after the level for nonlinear behavior (a₄).

${R = {a_{0} + \frac{a_{1}*d^{2}}{a_{2}^{2} + d^{2}}}},$where R is the mean discharge rate, and d is

$d = \left\lbrack \frac{a_{3}^{({\frac{1}{a_{4}} - 1})}*p^{\frac{1}{a_{4}}}}{a_{3}^{({\frac{1}{a_{4}} - 1})} + p^{({\frac{1}{a_{4}} - 1})}} \right\rbrack^{a_{4}}$where the variables denote the following:

-   -   a₀=the spontaneous discharge rate of the primary afferent,    -   a₁=the maximum increase of the discharge rate,    -   a₂=the sound pressure of the half maximum discharge rate,    -   a₃=the sound pressure at which nonlinear behavior occurs,    -   a₄=the exponent of the power-law slope in the nonlinear region,        p the sound pressure level at the tympanic membrane, and

p=10*log 10(abs(S1(frequency))), where S1 is the Short Time FourierTransform (STFT) of the acoustic signal.

Each of FIGS. 3A-3E illustrates the mean discharge rate R having variousvalues of the parameters a₀, a₁, a₂, a₃, a₄. Values of each parameterper graph of FIGS. 3A-3E are provided in Table 1 below.

TABLE 1 Parameter values for FIGS. 3A-3E a₀ a₁ a₂ a₃ a₄ FIG. 3A 0:0.1:11 20 50 0.5 FIG. 3B 0 0:0.1:1 20 50 0.5 FIG. 3C 0 1 5:5:50 50 0.5 FIG.3D 0 1 20 20:10:120 0.5 FIG. 3E 0 1 20 50 0.1:0.1:1

Traces in FIG. 3A show that the spontaneous discharge rate a₀ shifts thecurve towards larger values. The maximum rate a₁ limits the maximum rateto the number selected (FIG. 3B). The level for threshold a₂ has largeeffects on the mapping. Low threshold values result in a fast increasein the rate and quick saturation whereas large threshold values slow theincrease in rate but limit the maximum in achievable rate (FIG. 3C).Smaller effects are seen from the parameters a₃ and a₄ (FIGS. 3D and3E). Default values are selected (a₀=0, a₁=1; a₂=20; a₃=50, and a₄=0.5),which must be adjusted individually during later sessions with the CIuser.

The above variables are examples of the types parameters that areadjusted during the audiologist tuning sessions. Any hearing device canhave more or fewer parameters noted above depending on the codingstrategy.

In the systems of the present application, the optimization system 200is used to optimize the values of the parameters of the coding strategyprogrammed on the memory 120 of the NSD 102. In the embodimentillustrated in FIG. 2 , the optimization system 200 is described asbeing embodied in first, second, and third modules 202, 204, 206. It isunderstood that any one or more of the three modules 202, 204, 206 canbe used independently or in any combination to describe the features andfunctions described herein. It is also understood that all three modules202, 204, 206 could be a single system, independent systems, orcombinations thereof. For example, a further embodiment of theoptimization system 1000 described below combines the first and secondmodules 202, 204 of the optimization system 200.

Referring to FIG. 2 , the first module 202 collects and analyzes a widerange of patient feedback 208 as input data to determine ranges for eachparameter tailored to the cochlea of the patient. The memory 130 on theoptimization device 126 includes stored instructions that, whenexecuted, cause it to prompt the patient to identify threshold decibellevels 210 under a plurality of conditions. In one embodiment, theplurality of conditions includes first through fifth conditionsdescribed below, although any number and/or types of conditions may beused to effectuate the desired thresholds.

The first condition determines the patient's threshold for detectingspeech. A sound is provided to the patient and gradually increases involume. The patient indicates when he or she first detects the noiseagainst a quiet background.

The second condition determines the patient's preference for the mostcomfortable decibel level. A sound bite of speech is provided to thepatient and gradually increases in volume. The patient indicates when heor she first understands the speech clearly at a comfortable level, suchas listening to an audiobook.

The third condition determines the patient's threshold for recognizingspeech. A sound bite of speech is provided to the patent at a highdecibel level and gradually decreases in volume. The patient indicateswhen he or she can no longer understand what is being said.

The fourth condition determines the patient's threshold for the mostuncomfortable decibel level. A sound bite of speech is provided to thepatient and gradually increases in volume. The patient indicates whenthe speech reaches a level that it is uncomfortable to hear.

The fifth condition determines the patient's threshold for understandingspeech while raising the signal to noise ratio. A sound bite of speechis played as the background noise is gradually increased (or the SNR isgradually decreased). The patient indicates when the speech is no longerrecognizable due to the background noise.

The GUI instructions 132 on the memory 130 of the optimization device126 provide algorithmic processing that compares the patient's thresholdlevels 210 for each of the five conditions with the threshold levels fornormal hearing listeners. The average levels of a normal hearinglistener are based on a database of audiological waves representingspeech having a variety of pitches and frequencies against variouslevels of background noise. If the threshold levels 210 are outside ofan acceptable range for each condition, the patient is deemed hearingimpaired. An output 212 of the first module 202 is a plurality of rangesof decibel levels the patient has indicated as being at an acceptablelevel or within an acceptable range per condition.

FIG. 4 provides an example user interface 300 for collecting audiometrydata 208, 210 in the first module 202. By striking the play button 302,the patient triggers the system to provide a sound. The user interface300 includes arrows 304 that the patient can select to modify thevolume. When testing other conditions, the arrows 304 may correspond tovariables other than volume as desired or necessitated by the testingcondition. The patient clicks on a button 306 labeled “accept” toidentify the decibel level that corresponds to the patient's preferencebased on the conditions above.

The first module 202 may be tailored to test for specific aspects of thecochlear implant NSD 102. For example, the threshold levels for thevarious conditions are tested for an auditory wave that is a complexsummation of many different wave forms that affect a plurality ofchannels of the electrode array. In some embodiments, the electrodearray of the cochlear implant is tested as a collective. In otherembodiments, the conditions are tested separately for each channel.

The second module 204 includes a plurality of user interfaces 500, 600,700 of FIGS. 5-8 , respectively, that prompt the patient to select apreferred set of data point(s) corresponding to a specific pitch and/orfrequency. The data points initially provided to the patient in thesecond module 204 are chosen by the second module 204 to be within thespecific ranges provided as output 212 from the first module 202. Withinthe second module 204, one or more user interfaces allow the patient tocompare a large number of parameter settings against one another.Through the use of the second module 204, the patient is presented withat least two sound options and asked to select the preferred option. Inresponse to receiving the user's preferred option, the system generatessubsequent sound options for testing. The system generates subsequentsound options based on user feedback related to the previous soundoptions. In a preferred embodiment, statistical analysis of theparameter space enables the system to select subsequent options that aremost likely to provide the most meaningful feedback to the system tooptimize the efficiency of the iterative selection process. For example,the statistical analysis may include the use of a Gaussian function.Accordingly, the system can automatically explore areas of the parameterspace that statistically will provide the most useful information, whichresults in the most efficient (though not necessarily straight line)path to optimal settings.

In the first embodiment shown in FIG. 5 , the user interface 400includes an area 402 where each dimension 404, 406 corresponds to aparameter belonging to a function that alters properties of eachauditory filter simultaneously. For example, in the embodiment shown inFIG. 5 , the parameters associated with the x- and y-axes 404, 406 maybe amplification and noise cancellation, respectfully. To begin thetuning process, a user selects a point within the two-dimensionalframework, area 402. The parameters are adapted to reflect the settingscorresponding to the selected point and a sound is presented to theuser. The user then selects another point within the area 402 and theparameters are updated and a further sound is presented to the userusing the updated parameters. The user continues selecting points in thearea 402, typically clustering within a zone within the area 402, untilthe user indicates a preferred setting by selecting the “accept” button408. Once a patient selects a specific space within the area thatreflects the preferred sound, that point is accepted and the task isreset with the new word and/or new parameters assigned to the x- andy-axes 404, 406 being appointed. After a plurality of trials have beencompleted, the points that were accepted are used to compute a bestestimate for the set of parameters being optimized and the parameters ofthe tested NDS 102 are updated accordingly.

In a second embodiment shown in FIGS. 6 and 7 , the user interface 500involves the implementation of an interactive genetic algorithm (IGA) todetermine ideal cochlear implant settings. Genetic algorithms use thebiological metaphor of evolution and natural selection to construct aset of rules for searching a parameter space for optimal solutions.Genetic algorithms are valued for their flexibility and robustness tolocal minima due to the high amount of stochasticity utilized during thesearch process.

In user interfaces 500, 600 the search is initiated by presenting thepatient with a small number of device parameters which he or she isasked to rate on a scale relative to each other. In one embodiment,about half of these initial parameters are drawn randomly and uniformlyfrom the parameter space while the other half are drawn at random withina parameter space closely related to the original device settings of thecochlear implant user. The relative ratings for each parameter are thenused as inputs for a fitness function which determines which of thesettings should be ‘selected’ to be recombined with other survivingparameters to create ‘child’ parameters that will then undergo the samepruning and recombination procedure in the next generation. Theseiterations proceed for about 15-20 generations at which point themajority of the recommendations made are appealing to the user.

For example, in the embodiment 500 shown in FIGS. 6 and 7 , the patientis instructed to comply with an instruction presented in the message box502 of the user interface 500. In the illustrated example, the messagebox 502 instructs the patient to “Use the panel on the right to indicatehow good or bad the current setting sounds.” The patient presses theplay button 504, which triggers a first sound to be presented to thepatient. The patient then selects either of the “good” button 506 or the“bad” button 508 provided adjacent to the message box 502 to provide arating that corresponds to a point on a rating spectrum 510 shown inFIG. 7 . In one example, the length of time that the button 506, 508 isselected corresponds to how strongly the sound is rated. For example,selecting the “good” button 506 for a single click corresponds to apoint on the spectrum 510 just to the right of the center 512, whileselecting the “good” button 506 for a longer period of time causes therating to be closer to the “good” end 514 of the spectrum 510. After aplurality of trials have been completed, the ratings provided by thepatient are used to compute a best estimate for the set of parametersbeing optimized and the parameters of the tested NDS 102 are updatedaccordingly.

In the third embodiment shown in FIG. 8 , the user interface 600 isbased on a machine learning framework known as the “dueling bandits”problem. Duels are defined as random comparisons between pairs ofparameters where the user determines the ‘winner’ of each duel. In theuser interface 600, two sets of device parameters are drawn at randomfrom the parameter space and played in sequence. The user then selectswhich of the two settings he or she liked more by pressing a left button602 or a right button 604 on the user interface 600, with an additionalbutton 606 to repeat the stimuli or ignore them if they both soundunacceptable or similar. The model works under the assumption that theparameters corresponding to the winners of these duels will, on average,be informative in defining a function for recommending sets ofparameters that have the highest probability of winning a duel against aparameter generated at random from the parameter space.

The second module 204, and the one or more user interfaces 500, 600, 700employed, provide specific parameter settings 222 associated withspecific sounds or environments.

Referring back to FIG. 2 , the third module 206 includes a database 224on which a reference bank of sounds or environments that identifyfrequency content associated with each acoustic environment is stored.Using wavelet scattering transforms, a classifier algorithm determinesthe frequency content of each acoustic environment or sound profile.Alternative methods such as traditional Fourier transforms orspectrograms may be used. Support vector machine (SVM), K-clusteringmechanisms, or any type of clustering methods is used to group theinformation contained in each acoustic environment to create a referencebank of scenarios (i.e., noisy restaurant, commuter train, office,living room, etc.).

Optimized parameter settings 222 associated with specific environmentsthat are output from the second module 204 are provided as input to thethird module 206. The optimized parameter settings 222 are matched toclusters within the reference bank of sounds in order to associate theparameter settings with a greater range of environments. Simultaneously,the acoustic environment received on the microphone or auditory inputsensor of the cochlea implant or other hearable device is compared withthe reference bank of sounds to identify a comparable environment havingassociated parameter settings. The associated parameter settings 230 areoutput to the memory of the cochlear implant and automatically factoredinto the coding strategy of the cochlear implant.

The third module controls the parameter settings of the auditory deviceor the NSD. In one embodiment, the auditory device 102 includes anauditory device processor 106, an auditory device output mechanismincluding one or more modifiable parameter settings, and an auditoryinput sensor 104 that detects an environmental sound and communicateswith the auditory device processor 108. The auditory device outputmechanism is any output mechanism of an auditory device, such as one ormore electrodes 112, 114 of a cochlear implant or a speaker on a hearingaid device. The auditory input sensor 104 may be a microphone positionedon the auditory device. The system also includes a database 123 ofreference sound profiles and a plurality of sets of parameter settings,each of which is paired with a corresponding sound profile. The database123 may be stored directly on the auditory device 192 or remotely on thepatient's mobile device or on a remote server.

The auditory device 102 also includes a memory 120 in communication withthe processor 108 and including instructions that, when executed by theprocessor, cause the processor 108 to undertake certain steps that matchthe environmental sound detected by the auditory input sensor 104 withthe reference bank of sounds 123 to identify a comparable environmenthaving associated parameter settings.

More specifically, the processor 108 first receives the environmentalsound detected by the auditory input sensor 104 and analyzes a frequencycontent of the environmental sound. The system may determine thefrequency content of the environmental sound by using a waveletscattering transform to analyze the frequency content of theenvironmental sound, using a Fourier transform to compute the frequencycontent of the environmental sound, or any other suitable classifieralgorithm to determine the frequency content of the acousticenvironment.

The processor 108 compares the frequency content of the environmentalsound with the sound profiles stored in the database 123. In response tothe comparison, the system selects one of the sound profiles andautomatically adjusts the parameter settings of the auditory deviceoutput mechanism, such as electrodes 112, 114, to match the set ofparameter settings associated with the selected sound profile. Each setof the plurality of sets of parameter settings may include amplificationsettings, compression settings, and directional noise rejectionsettings.

FIG. 9 is an example of the clustering 900 of different sound profiles902 based on a plurality of variable parameters 904 to create a databaseor reference bank of environments and corresponding parameters 906.Example sound profiles of environments include the beach, the bus, thecity, the forest, the office, a specific person's voice, such as aparent or child, or other easily characterized or recognizableenvironments or sounds. The database also includes a set of parametersettings associated with each sound profile. The database may alsoinclude parameter settings associated with combinations of recognizedenvironmental sounds, such as, for example, the recognition of aspouse's voice in a home living room environment, which may be adifferent setting than a child's voice in the same living room setting,which may be different from either of the voices in a dining roomsetting. During use, the system determines one or more properties of thefrequency content of the environmental sound such as, but not limitedto, a signal-to-noise ratio, an amplitude range, and a pitch range torecognize the appropriate saved parameters to apply to the system. Whileonly three variable parameters 904 are illustrated in FIG. 9 , theclustering software may use many more than three or as few as oneparameter to cluster the sounds by similarity.

Each sound profile may also be associated with a stored geolocation. Alocation sensing mechanism in communication with the auditory deviceprocessor determines the present geolocation of the auditory device.After the system selects a sound profile that corresponds to theenvironmental sound, the processor may further compare a presentgeolocation of the auditory device output mechanism identified by thelocation sensing mechanism with the stored geolocations. The geolocationmay identify a subset sound profile with an associated set of parametersettings. The geolocation may be particularly useful in maintainingconsistency in settings as there are times the positional location willbe more stable than the sound environment. As such, it may be the casethat based on a given geolocation, the processor is instructed to onlychoose between a limited number of settings. For example, in the“office” geolocation, the processor may be restricted to choosingbetween the (i) office desk, (ii) office conference room, and (iii)office cafeteria, settings. A more complex application may includerecognizing the geolocation (for example, the user's home), which limitsthe possible sound profiles from which to choose, then recognizes thebackground noise (for example, the user's living room with thetelevision on) then recognizes the user's spouse's voice to apply asound profile matching settings optimized for the user to hear theuser's spouse in the user's living room with the television on in thebackground.

If the acoustic environment received by the microphone does notcorrespond closely with any of the reference bank of sounds 906, a newenvironmental setting may be created. In one embodiment, the patientcould update his or her parameter preferences for the new acousticenvironment either through the hearing device itself or using a mobileapplication associated with the optimization system of the presentapplication, either through a phone or tablet connected to his or herhearing aid or cochlear implant. In some embodiments, the first andsecond modules are accessible by the patient through a mobileapplication on a mobile device. The patient can use the mobileapplication to tune the parameters to the present environment and storethe set of parameter settings associated with the specific environmentalsound profile in the database of sound profiles 906.

The patient may also add to the reference bank of sounds associated withspecific parameter settings by simulating the sounds during thepatient's visit to an audiologist. For example, an audiologist wouldplace a hearing-impaired user in a sound booth and play speech-in-noiseor speech-in-babble or even more specific acoustic environments, such asspeech on an airplane or speech-in-wind. Using the second module of theoptimization system, the patient sets his or her preferred parameters.When the patient is in the real-world environment, all parametersettings are updated based on the current environment's similarity tothe previously simulated environments.

Referring to FIG. 10 , an alternative embodiment of the optimizationsystem 1000 includes a module 1002 that tests a patient's hearing byanalyzing patient feedback 1004 as input data on a real-time basis todevelop an output 1006 that is used to tune the NSD 102. The memory 130on the optimization device 126 includes stored instructions that, whenexecuted, cause it to analyze a patient's feedback 1004 in real-timethrough a Gaussian process in order to select a subsequent tone fortesting that will provide the most useful information for determiningthe patient's hearing thresholds graph as the output 1006. The output1006 is then provided as input to tune the hearing device 102 once thetesting is complete. In the illustrated embodiment, the hearing device102 is a hearing aid, although other devices such as headphones orcochlear implants may be used as described in greater detail below.

FIG. 11 provides an example user interface 1008 for collectingaudiometry data in the module 1002. By pushing the start button 1010,the patient triggers the optimization system 1000 to provide one of aplurality of tones, each having an associated frequency and intensity(decibel level), through one of the right and left ear pieces. Thepatient selects one of a left button 1012 and a right button 1014 toidentify in which ear he heard the tone, if at all. The selection ornon-selection of the left button 1012 or the right button 1014 is thepatient feedback 1004 that is analyzed by the optimization system 1000to select the subsequent tone to be tested.

In contrast to a conventional hearing test that cycles through a setnumber of frequencies to test the patient's hearing thresholds, theoptimization system 1000 simultaneously tests both the right and lefthearing at random frequencies, utilizing a Gaussian process to compute astatistical distribution based on the patient feedback 1004, identifythe area of the audiogram most in need of data, and select a subsequenttone to test on the patient that will provide the information mostuseful for the area in need. Simply put, each data point of the patientfeedback 1004, or each of the patient's selection or non-selection ofthe left button 1012 or the right button 1014, provides some insightinto the area surrounding that specific data point. The optimizationsystem 1000 generates a statistical distribution of the frequency andvariance of each data point, identifies the highest point on thevariability function (i.e., the highest variability level, or the lowestconfidence level) of the distribution, and plays a subsequent toneassociated with that highest point of variability of the previous tone.In this way, the system 1000 selects a subsequent tone that provides themost information possible in order to efficiently determine theboundaries of the patient's hearing curve. The patient then responds tothe subsequent tone by either selecting or not selecting the left button1012 or the right button 1014, and the new data point is added to thedata set to be processed and analyzed by the optimization system 1000 asdescribed above.

FIGS. 12-15 illustrate a user interface 1100 that illustrates the dataset as it is collected over time by the optimization system 1000. Theuser interface 1100 includes four graphs for each of the right and leftears: a tone graph 1102, a threshold estimate graph 1104, a GaussianProcess frequency (GPμ) graph 1106, and a Gaussian Process variance(GPa) graph 1108. The user interface 1100 also indicates the totalnumber 1110 of tones that have been tested as well as the cumulativeconfidence level 1112 for each of the patient's right and left devices.A hearing thresholds graph 1114 is updated in real-time as patientfeedback 1002 is collected. The user interface 1100 also includes astatus indicator 1116, noting that the optimization system 1000 is readyor that testing is active, paused, or completed.

The user interfaces 1100A-D illustrate the collection of data over aperiod time during a patient test. FIG. 12 illustrates the userinterface 1100A before any data is collected. FIGS. 13 and 14 illustratethe user interface 1100B and 1100C after 13 and 34 data points,respectively, are collected. FIG. 15 illustrates the user interface1100D after 70 data points are collected and the hearing thresholdsgraph 1114D is finalized.

Best seen in FIGS. 13-15 , the tones graph 1102B-1102D shows each tonethat has been tested and whether the patient heard the tone correctly.An “0” indicates that the patient correctly heard the tone, and an “X”indicates that the patient did not hear the tone.

The threshold graph 1104A-1104D includes an upper curve and a lowercurve that correspond to the tested tones data as shown in the tonesgraph 1102. The space above upper curve corresponds to the tones thatthe patient can hear, or the “X” marks, while the space below the lowercurve corresponds to the tones that the patient cannot hear, or the “0”marks. The space between the upper and lower curves is an unknown areathat the optimization system seeks to minimize. In comparing thethreshold graphs 1104B and 1104D, the space between the upper and lowercurves is large when few data points have been collected in FIG. 13 ,and is minimized once the test is completed in FIG. 15 .

The GPμ graph 1106A-1106D illustrates the likelihood of hearing eachtone, or each frequency at varying intensities, of the tones of the testdata collected. The white space corresponds to tones that the patientcan hear, and the black space corresponds to tones that the patientcannot hear. The gradations between the black space and the white spacecorrespond to a transition area between what is known and what isunknown, which the optimization system is minimizes as data iscollected. In comparing the GPμ graphs 1106B and 1106D, the gradationarea between the white and black spaces is wide when few data pointshave been collected in FIG. 13 , and is minimized once the test iscompleted in FIG. 15 .

The GPσ graph 1108A-1108D illustrates the variances of the tones of thetest data collected. The white space corresponds to tones that thepatient can hear, and the black space corresponds to tones that thepatient cannot hear. The gradations between the black space and thewhite space correspond to a transition area between what is known andwhat is unknown, which the optimization system is minimizes as data iscollected. In comparing the GPμ graphs 1106B and 1106D, the gradationarea between the white and black spaces is wide when few data pointshave been collected in FIG. 13 , and is miminized once the test iscompleted in FIG. 15 .

Once the confidence level has reached a certain minimum level, the testis completed and the hearing thresholds graph 1114 is ready to be inputinto the hearing device 102 for tuning. The optimization system 1000 mayalso include an upper threshold that corresponds to a minimumunacceptable level. If a patient surpasses the upper threshold, the testis invalid.

The NSD 102 may be any hearing device, such as a hearing aid, a cochlearimplant, and headphones. In one embodiment, the patient connects hishearing aids to his mobile device, and runs the optimization system 1000via a mobile app on his mobile device in order to tune the hearing aids.The patient could update the audiogram at any time to automatically tunethe hearing aid. In other embodiments, a patient may use theoptimization system through a kiosk at the point of sale of the hearingaid, headphones, or other hearing device. In that example, the patientwould test his hearing at the kiosk and the audiogram from theoptimization system would be used to tune the hearing device at the timeof purchase. Each of the above systems may include features thatcalibrate to the specific test device.

The foregoing description merely illustrates the principles of thedisclosure. Various modifications and alterations to the describedembodiments will be apparent to those skilled in the art in view of theteachings herein. It will thus be appreciated that those skilled in theart will be able to devise numerous systems, arrangements and methodswhich, although not explicitly shown or described herein, embody theprinciples of the disclosure and are thus within the spirit and scope ofthe present disclosure. From the above description and drawings, it willbe understood by those of ordinary skill in the art that the particularembodiments shown and described are for purposes of illustrations onlyand are not intended to limit the scope of the present disclosure.References to details of particular embodiments are not intended tolimit the scope of the disclosure.

We claim:
 1. An optimization system for testing a patient's hearingcomprising: a controller; an ear piece in audible communication with thecontroller; a memory in communication with the controller and includinginstructions that, when executed by the controller, cause the controllerto: provide a series of tones to the ear piece; receive feedback fromthe patient between each tone provided, wherein each feedback indicatesthat the respective tone was detected in the ear piece or that therespective tone was not detected; generate a data point to be used in anaudiogram after receiving each feedback, wherein each data point isbased on the respective feedback; after each data point is generated,compute a statistical distribution based on the generated data points;identify an area of the statistical distribution most in need ofadditional data; and select a subsequent tone to provide in the seriesof tones, wherein each subsequent tone provided in the series of tonesis a tone represented in the area of the statistical distribution mostin need of additional data at the time of selection.
 2. The optimizationsystem of claim 1, wherein the ear piece comprises a right ear piece anda left ear piece, and the controller is configured to: provide a seriesof tones to the right and left ear pieces, wherein each tone is providedto one of the right and left ear pieces, and generate a data point to beused on one of a right audiogram and a left audiogram after receivingeach feedback, wherein each data point is based on the respectivefeedback.
 3. The optimization system of claim 1, wherein the controllerutilizes a Gaussian process to compute the statistical distribution. 4.The optimization system of claim 1, wherein the controller is configuredto generate a hearing thresholds graph including an upper curve and alower curve, each of the upper and lower curves based on the series oftones and the feedback from the patient.
 5. The optimization system ofclaim 4, wherein an upper area of the hearing thresholds graph above theupper curve corresponds to one or more tones that were detected.
 6. Theoptimization system of claim 5, wherein a lower area of the hearingthresholds graph below the lower curve corresponds to one or more tonesthat were not detected.
 7. The optimization system of claim 6, wherein acentral area of the hearing thresholds graph between the upper and lowercurves corresponds to one or more tones that were not provided fordetection.
 8. The optimization system of claim 6, wherein the upper andlower curves align after the series of tones has been provided.
 9. Theoptimization system of claim 4, wherein the controller is configured todisplay a graphical user interface including the audiogram and thehearing thresholds graph.
 10. The optimization system of claim 9,wherein the graphical user interface includes a right cumulativeconfidence level and a left cumulative confidence level.
 11. Theoptimization system of claim 1, wherein the controller is furtherconfigured to: provide a button to be selected by the patient after eachtone is provided, wherein selection or non-selection of the buttoncomprises the feedback from the patient that indicates whether therespective tone was detected or not detected.
 12. The optimizationsystem of claim 11, wherein the controller is further configured todisplay the button for selection through a graphical user interface. 13.The optimization system of claim 1, further comprising a hearing device,and wherein the controller is configured to output the generated datapoints to the hearing device.
 14. A method of testing a patient'shearing comprising the steps of: providing a series of tones to an earpiece; receiving feedback from the patient between each tone provided,wherein each feedback indicates that the respective tone was detected inthe ear piece or that the respective tone was not detected; generating adata point to be used on an audiogram after receiving each feedback,wherein each data point is based on the respective feedback; after eachdata point is generated, computing a statistical distribution based onthe generated data points; identifying an area of the statisticaldistribution most in need of additional data; and selecting a subsequenttone to provide in the series of tones, wherein each subsequent toneprovided in the series of tones is a tone represented in the area of thestatistical distribution most in need of additional data at the time ofselection.
 15. The method of claim 14, wherein each tone in the seriesof tones is provided to one of a right ear piece and a left ear piece,and wherein each data point is generated to be used in one of a rightaudiogram and a left audiogram after receiving each feedback.
 16. Themethod of claim 14, further comprising the step of: providing a buttonconfigured to be selected by the patient after each tone is provided,wherein the selection or non-selection of the button comprises feedbackfrom the patient that indicates whether the respective tone was heard.17. The method of claim 16, further comprising the step of displayingthe button through a graphical user interface.
 18. The method of claim14, further comprising the steps of: generating a hearing thresholdsgraph based on the feedback and including at least one curve separatingthe graph into an upper section and a lower section.